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Teachers were trained in either or both of two 
versions of Effective Classroom Management — Junior High (ECM). m the 
second year of the study, teachers were taught techniques in 
discipline, communication, and self-esteem enhancement. The goals of 
the in-service training were to make teachers more responsive to 
students' affective and cognitive needs. The short-term goals of 
implementation were to foster positive student attitudes, behaviors 
and norms regarding self, peers, and school. These changes were 
expected eventually to lead to reduced acceptance and use of 
psychoactive substances. The present report evaluates the second year 
of ECM. The research design employed a nonequivalent control group 
with a pretest at the beginning of the first year of the study and a 
posttest at the end of the second year. The treatment group consisted 
of 8th and Sth grade students and teachers from a junior high school. 
Most of the teachers in this school participated in at least one year 
of ECM training. The control group consisted of 8th and 9th grade 
students and their teachers in another junior high school in the same 
school district. The predicted positive effect on satisfaction with 
teaching was significant for all participants, but not all 
experimental teachers. ECM did not appear to affect student outcomes. 
(Author/GK) 
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Abstract 



This study evaluated the cumulative effects on teachers and students 
of two years of Effective Classroom Management— Junior High (ECM) . Teachers 
were trained in either or both of two versions of ECM. The components of 
both versions of ECM are similar to those in many affectively-oriented in- 
service courses. In the first year, teachers were taught techniques in 
problem-solving, communication, and self-esteem enhancement. In the second 
year, teachers were taught techniques in discipline, communication, and 
self-esteem enhancement. The goals of the course were to make teachers 
more responsive to students 1 affective and cognitive needs, thereby foster- 
ing positive attitudes, behaviors, and norms regarding self, peers, and 
school. The ultimate objective was to reduce students 1 acceptance and use 
of psychoactive substances. 

The study employed a nonequivalent control group design with pretest 
and posttest. The experimental group was composed of 464 students enrolled 
in one junior high school, along with 38 of their teachers. The control 
group consisted of 264 students in another junior high, and their teachers. 
Several teacher outcomes were measured: satisfaction with teaching, faculty 
cohesiveness, and effectiveness at achieving ECM-related teaching* objectives. 
Data were collected from students regarding a number of mediating variables 
(e.g., self-esteem, attitudes toward school, perceptions of peers 1 norms and 
behaviors) and drug-specific variables (e.g., current use, lifetime use, 
intentions to use in the future, attitudes toward use, perceived benefits 
and adverse consequences of use). Student achievement and attendance data 
were gathered from school district records. 

Process evaluation data included a) teacher feedback on the individual 
training sessions, b) questionnaire surveys of teachers at the end of training 
and the end of the school year, and c) observations of classroom implementation. 
These data showed that the training was regarded very highly by the teachers, 
but that they did not regularly use most of the ECM skills in their classrooms. 
Overall, levels of implementation of skills were disappointing, though the 
teachers varied considerably in how often they used the skills. 

Comparisons of experimental and control group teachers revealed no treat- 
ment effects on effe;tiveness at achieving ECM objectives or on faculty 
cohesiveness. However, a significant positive effect was found on satisfaction 
with teaching for the teachers who completed the in-service training, but not 
for all teachers in the experimental school. 

The predicted pattern of student outcomes was not obtained. Althouoh 
hypothesized effects were found for ninth grade males on perceived teaching 
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climate and locus of control, these effects did not form an interpretable 
pattern. Nor was any pattern of effects obtained for eighth grade males 
or females or ninth grade females. 

The relationship between the students 1 posttest outcomes and the number 
of ECM-trained teachers to which the students had been exposed was examined 
within the experimental school. No relationship was found between student 
outcomes and the amount of exposure to ECM teachers. This finding provided 
further evidence that the few treatment effects that did obtain were probably 
spurious. 

Thus, ECM failed to produce a consistent pattern of positive effects, and 
the failure is best attributed to weaknesses in the ECM in-service curriculum 
and the training procedures. 
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Introduction 



Affective teacher training courses have been a major component of 
many primary prevention programs. State-wide prevention programs in 
Michigan (Michigan Department of Education, Note 1), Georgia (Georgia 
Department of Human Resources, Note 2), and Pennsylvania (Bandt, Hammond, 
Wisdo & Mitzel, Note 3) have emphasized such training. Affective teacher 
training programs have been shown to be effective at improving peer 
relationsnips, classroom discipline, attitudes toward school, and academic 
achievement (Baskin and Hess, 1980). Therefore, training "significant 
others" (e.g., parents, teachers) in affective skills may be an effective 
approach to prevention (Schaps, DiBartolo, Moskowitz, Palley & Churgin, 1981). 

In the present study, teachers were trained in either or both of two 
versions of Effective Classroom Management— Junior High (ECM). The components 
of both versions of ECM are similar to those in many affectively-oriented in- 
service courses. In the first year of this study, teachers were taught 
techniques in communication, problem-solving and self-esteem enhancement. 
In the second year of the stud^ , teachers were taught techniques in discipline, 
communication, and self-esteem enhancement. The goals of the in-service 
training were to make teachers more responsive to students' affective and 
cognitive needs. The short-term goals of implementation were to foster 
positive student attitudes, behaviors and norms regarding self, peers, and 
school. These changes were expected eventually to lead to reduced acceptance 
and use of psychoactive substances. 
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Participating teachers applied the in-service skills in their class- 
rooms under the guidance cf the trainers. The teachers were thus the 
critical links in the delivery of this intervention; the adequacy of the 
"treatment" depended upon teachers' use of the techniques. Consequently, 
teachers 1 reactions to the training and their use of the ECM skills were 
evaluated intensively. 

The effects of the first year of ECM training have been evaluated pre- 
viously (Schaps, Moskowitz, Condon, Malvin & Schaeffer, Note 4). Treatment 
effects on 7th-9th qrade teachers or students generally were not found. The 
process evaluation revealed poor implementation of most skills, especially the 
problem solving skills. Therefore, in the second year of the ECM training, 
problem solving skills were eliminated, and discipline skills were added. 
(Detailed documentation of the curriculum is available [Adams, Slimmon & Schaps, 
Mote 5]). 

The present report evaluates the second year of ECM. The research design 
employed a nonequivalent control group (Cook and Campbell, 1979) with a pretest 
at the beginning of the first year of the study and a posttest at the end of 
the second year. The treatment group consisted of 8th and 9th grade students 
and teachers from a junior high school. (The students were in 7th and 8th 
grade in the first year of the study J Most of the teachers in this school 
participated in at least one year of ECM training. The control group consisted 
of 8th and 9th grade students and their teachers in another junior high school 
in the same school district. 

Various mediating and drug-specific outcome variables were measured in 
this study (see Methods section). We hypothesized that ECM would positively 
impact perceived classroom climate, attitudes toward school, social self-esteem, 
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5. 

and discipline behaviors, because the in-service skills bear most directly 
on these variables. 

During the next year, the teachers in the treatment school will receive 
additional training in ECM skills, and final posttest data will be obtained 
for the 8th grade students. 
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Method 

Assignment of Schools to Condition 

Two junior high schools (grades-7-9) from a predominantly white, middle- 
class, suburban public school system in Northern California participated in 
this study. One school was assigned to the treatment condition and the other 
school to the control condition. All teachers in the treatment school were 
offered two years of ECM in-service training; the control school received 
no service deliveryl 

Subjects 

All students and their teachers in the treatment school constituted the 
treatment group. 1 Of the 61 treatment school faculty members, 15 (25%) com- 
pleted only the first year of ECM training, seven others (11%) completed only 
the second year of training, and 16 others (26%) completed both years of 
training. 

At the beginning of the study there were 725 students in grades 7 and 8 
in the treatment school and 460 in the control school. 2 A total of 248 
students, (34%) from the treatment schoe-1-and 180 students (39%) from the 
control school were excluded from the study because two complete years of 

'Special education students and teachers in both schools were not 
included in the study. 

2 An additional 142 students from the treatment school and 256 students 
from the control school were excluded from the present study because they 
participated in other studies. 
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data were not available for them. In addition, 13 treatment school students 
and 16 control school students were eliminated because they reported repeated 
use of a bogus drug . 

The treatment group in the analyses of the second year data included 
464 (64% of total enrollment) students, with 115 males and 104 females in 
grade 8 and 128 males and 117 females in grade 9. Analyses for the control 
group included 264 students (57% of enrollment), with 50 males and 54 females 
in grade 8 and 75 males and 85 females in grade 9. The ethnic composition of 
the analysis sample was 93% (N = 432) white in the experimental and 87% (N = 
230) white in the control school. Of the total sample, 9% (N = 66) were 
minority students, with Mexican-Americans (N = 32) and Asian/Pacific Islanders 
(N = 17) comprising the largest subgroups. 

In-Service Training Program 

The second year of the ECM in-service program consisted of seven two- 
hour training sessions held weekly after school. There were also two 
"reunion" sessions in February and March, 1980. 3 The teachers were paid $200 
for attending the sessions and trying to apply the ECM skills in their class- 
rooms. They also were offered graduate-level credit from a local university 
for completing the training. 

Both trainers were experienced in conducting in-service programs for 
teachers. They also had been classroom teachers and school administrators, 
and had taught. many of the ECM skills in previous training programs. 

3 The first-year course consisted of ten two-hour sessions plus one 
"reunion" session. 
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The training sessions generally followed a standard format. First, 
a self-concept enhancement technique or activity was introduced. Then the 
skills taught in prior sessions were reviewed and the teachers' experiences 
using the skills during the previous week were discussed. After this, new 
skills were introduced and practiced. 

Four communication skills were taught during the first two training 



sessions: 



t I-Messages— a technique for effectively communicating the 
impact of another's behavior upon oneself; 

0 Clarifying Responses-brief questions -and comments by which 
a listener can prompt further thinking by a speaker; 

t Reflecting Feelings-techniques that enable a listener to 

indicate acceptance and accurate understanding of the speaker's 
feelings; and 1 

t Reflecting Content— techniques that enable a listener to show 
interest and to indicate understanding of the speaker's message. 

In the next four sessions teachers were taught a sequence of classroom 
■nanagement skills for minimizing and handling discipline problems. Techniques 
were taught for effectively communicating expectations and establishing class- 
room rules. In addition, skills were taught for managing discipline problems 
and for rewarding desired behaviors. These skills included systematic positive 
reinforcement of appropriate student behavior: 

0 Positive Nonverbal Cues (e.g., smile, nod); 

t Positive Verbal Feedback (e.g., praise); and 

0 Tangible Reinforcers (e.g., token system); 
and systematic discipline of inappropriate behavior: 



o ] 0 

ERIC 



9. 



0 Negative Nonverbal Cues (e.g., eye contact, touch); 

0 Negative Verbal Feedback (e.g., request for behavioral change); and 

0 Time-out (e.g., isolation). 

Activities to enhance student self-concept were introduced at five 
sessions and were the major component of the last session. Activities were 
taught that facilitate classroom discussion of students' experiences, talents, 
and achievements, and that elicit positive feedback from peers. Another 
approach to improving self-concept included modifying the regular curriculum 
to help students organize their work, assist each other, and gain recognition 
for their efforts. 

One of the trainers observed the teachers in their classrooms at least 
three times. These visits began during the training and continued until May, 
1980. After each visit, the trainer met briefly with the teacher to provide 
feedback on the teachers' classroom behavior. 

Survey Administration Schedule and Procedures 

Student survey. The student pretest was administered in October 197H, 
posttest 1 in May 1979, and posttest 2 in May 1980, by four substitute teachers 
trained in survey administration procedures. The questionnaires were adminis- 
tered during two regular classes. The Student Questionnaire and the Self 
Observation Scales were administered during the first session, and the Drug 
and Alcohol Survey (DAS) was administered during the second session. Make-up 
sessions were held for students who were absent at the time of the original 
sessions. 
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The administration procedure stressed confidentiality. Students were 
identified by their school district identification numbers. Questionnaires 
were pre-labeled with student names on the cover sheet and student identifi- 
cation numbers on page one. In a prepared statement, administrators assured 
students of complete confidentiality and explained the need for identification 
numbers as a way of tracking students over time. For the DAS administration, 
student were instructed to tear off thr cover page that displayed their names 
to further enhance the confidentiality induction. 

Teacher survey. The teacher pretest was administered in October 1978, 
posttest 1 in May 1979, and posttest 2 in May 1980, during faculty meetinos 
at each school. To ersure the privacy of their responses, teachers were 
provided with questionnaires containing unique identifiers. 

Student Self-Report Measures 

Student pretest self-report data were obtained with the Self Observation 
Scales (SOS) (Stenner and Katzenmeyer, Nc\> 6), the Intellectual Achievement 
Responsibility Questionnaire (IAR) (Crandall , Katkovsky, and Crandall, 19GG), 
and the Drug and Alcohol Survey (DAS) (Moskowitz, Schaeffer, Condon, Schaps 
and Malvin, Note 7). The SOS is a nationally normed instrument which measures 
students' perceptions of themselves and their relationships to their peers, 
their teachers, and their school. The IAR measures the belief in one's own 
control over intellectual and academic performance." The DAS assesses for 

'Four items referring to "parents" were adapted to read "an adult who 
knows you in order to conform to California Education Code. When passive 
parental permission for student participation is obtained researchers cannot 
ask questions aoout family life. 
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each of ten substances: the students' lifetime and current use, their 
attitudes toward use, their intentions to use, and their perceptions of 
peers' attitudes toward drug use. 3 In addition, the DAS contains measures 
of general drug attitudes, and the perceived benefits of using three dif- 
ferent substances: alcohcl, cigarettes, and "pills 6 ." 

Posttest 1 data were obtained with the SOS, the Student Questionnaire 
(SQ) (Moskowitz, Condon, Brewer, Schaps and Malvin, Note 8) and a revised 
version of the DAS. The SQ contained a) 18 items from the IAR; b) the 
Scholastic subscale from the secondary level of the Self Appraisal Inventory 
(Instructional Objectives Exchange, Note 9), a criterion-referenced measure 
of academic self-esteem; c) the Authority and Control and Interpersonal 
Relationships with Pupils subscales from the secondary level of the School 
Sentiment Index (Instructional Objectives Exchange, Note 10), a criterion- 
referenced measure of attitudes toward school; and d) a measure of perceived 
peer attitudes toward school developed for this study by adapting 11 items 
from eight instruments that measure attitudes toward school. The revised DAS 
assessed for each of ten substances: the students' lifetime and current use, 
their attitudes toward use, their intentions to use in the future, and their 
perceptions of peers' attitudes toward use and the prevalence of peer use. 7 

5 The substances were alcohol, cigarettes, marijuana or hashish, inhalants, 
barbiturates or tranquilizers, amphetamines or stimulants, cocaine, PCP, LSD 
or psychedelics, and heroin or morphine. "Street" names were provided for 
most substances. Current use was defined as "during the past three months." 
Intentions were defined as "during the next two years." Peers were defined as 
"most students in my grade." 

6 Pills were defined as "pep pills, sleeping pills, uppers, downers, soapers. 

7 Current use was defined as "during the last four weeks." Intentions were 
defined as "during the next year." 
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In addition, the revised DAS measured drug knowledge, general drug attitudes, 
and the perceived positive (benefits) and negative (costs) consequences of 
using three different substances. Some item wordings and response formats 
differed from the earlier version. 

Posttest 2 data were collected with a new version of the SQ and the 
revised DAS. The new SQ incorporated empirically derived scales from the 
original SQ and the SOS (Moskowitz, et al., Note 8). Unlike the original SQ, 
the new SQ employed a four-point Likert response format for most items. 

The measures utilized for data analysis were derived from empirical 
scaling. The details of the scaling procedures and results have been reported 
earlier (Moskowitz, et al., Notes 7 and 8). The final pretest and posttest 
scales appear in Table 1, which lists the number of items contained in each 
scale., and the scale's internal consistency reliabilities estimated by 
coefficient alpha. The final scales included pretest and posttest measures 
of locus of contro 1 for success, locus of control for failure, academic self- 
esteem, social self-esteem, affective teaching climate, and attitudes toward 
school. The drug-related scales included several measures of drug attitudes; 
perceived benefits of alcohol, marijuana, and pill use; attitudes toward "soft" 
and "hard" drug use 9 , perceived peer attitudes toward soft and hard drug use; 
and involvement in use for each of the ten substances 9 . Other post-test 
measures were perceived peer attitudes toward school; perceived costs of 
alcohol, cigarette, and pill use; and drug knowledge. The reliabilities 

3 The "soft" substances included alcohol, cigarettes, and marijuana; the 
"hard" substances included the other seven drugs. 

9 The involvement scales consisted of items assessing current use, 
lifetime use, and intentions to use. 
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Table 1 

student pretest and posttest scales, number of items and 
internal consistency reliabilities (coefficient alpha) 



Academic Self-Esteem 
(Acad Self) 



PRETEST POSTTEST 
Number of Number of 



Subscale N Items Reliability N Items Reliability 

1944 13 .63 551 7 .66 



Locus of Control: Success 
(Control Sue) 

Locus of Control : Failure • 

(Control Fail) 1944 10 .61 551 7 .61 



513 5 .65 551 12 .84 



Social Self-Esteem * • 

(Social Self) 513 11 .80 551 11 .80 

Affective Teaching Climate 

(Af fee Climate) 513 8 .79 551 18 .89 

Attitudes Toward School 

(Att School) 513 8 .82 551 8 .83 

Perceived Peer Attitudes Toward 

School n __, n -,, 

(Peer Att Sch) - NA - 551 8 .74 

Drug Knowledge , 

(.Knowledge) - NA - 586 7 .40 

General Drug Attitudes 

(General Att) 473 17 .92. 586 17 .93 

Attitudes Toward Soft Drug Use 

(Soft Att) 473 3 .79 586 3 .76 

Attitudes Toward Hard Drug Use 

(Hard Att) 473 7 .93 586 7 .93 

Perceived Peer Attitudes Toward 

Soft Drugs „ 

(Soft Peer Att) 473 3 .87 586 ■ 3 .84 

Perceived Peer Attitudes Toward 
Hard Drugs 

(Hard Peer Att) 473 7 ,96 586 7 .So 

Perceived Benefits of Alcohol Use 

(Ale Benefits) 473 8 .85 586 8 -8d 

Perceived Benefits of Marijuana Use 

(Pot Benefits) 473 8 .89 586" 8 .91 
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PRETEST 



POSTTEST 



Subscale 

Perceived Benefits of Pill Use 
(Pill Benefits) 

Perceived Costs of Alcohol Use 
(Ale Costs) 

Perceived Costs of Marijuana Use 
(Pot Costs) 

Perceived Costs of Pill Use 
(Pill Costs) 

Perceived Peer Use of Soft Drugs 
(Soft Peer Use) 

Perceived Peer Use of Hard Drugs 
(Hard Peer Use) 

Involvement in Alcohol Use 
(Ale Involve) 

Involvement in Cigarette Use 
(Cig Involve) 

Involvement in Marijuana Use 
(Pot Involve) 

Involvement in Inhalant Use 
(Inh Involve) 

Involvement in Barbiturate Use 
(Barb Involve) 

Involvement in Amphetamine Use 
(Amp Involve) 

Involvement in Cocaine Use 
(Coc Involve) 

Involvement in PCP Use 
(PCP Involve) 

Involvement in LSD Use 
(LSD Involve) 

Involvement in Heroin Use 
(Heroin Involve) 



Number of 

Items Reliability N 



Number of 

Items Rel i abi 1 i 



473 



473 

473 

473 

473 

473 

473 

473 

473 

473 

473 



8 

NA 

NA 

NA 

NA 

NA 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 



.92 



.92 

.92 

.95 

.86 

.84 

.88 

.75 

.83 

.70 

.93 



586 

586 

586 

586 

586 

586 

586 

586 

586 

586 

586 

586 

586 

586 

586 

586 



8 
5 
5 
5 
3 
7 



3 

3 

3 

3 

3 

3 

3 

3 

3 



.91 

.84 

.90 

.89 

.82 

.96 

.88 

.91 

.95 

.70 

.89 

.90 

.88 

.86 

.87 

.77 
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obtained were adequate for all scales except drug knowledge. 10 Pretest and 
posttest reliabilities for equivalent scales were highly consistent despite 
some differences in item wordings and response formats. 

Student archival outcome data . Student discipline records for four months 
in the spring semesters of 1979 and 1980 were obtained from both schools. 
These records were maintained by the deans in the two schools to document 
student behavior problems. Two indices were constructed for each year: a) a 
drug behavior problem index (Drug Problems) consisting of categories involving 
use, possession or sale of tobacco, alcohol, marijuana and other drugs; and 
b) a general behavior problem index (Non-drug Problems) containing categories 
for all other types of student misbehavior. 

Other student records were obtained from the school district. The total 
number of unexcused absences for each school year (Unex Abs) was used as a 
measure of student attendance. This type of absence occurs when a student 
does not provide the school with a parental excuse indicating that the student 
was sick. Grade point average (GPA) for each spring semester, where academic 
grades from all courses were weighted equally, served as a measure of academic 
achievement. An index of exposure to the ECM- trained teachers (ECM Fxpos) 
v/as created for students in the experimental school. This index equalled the 
the number of courses in which the student was enrolled that were taught by 
teachers trained in ECM. Finally, students 1 sex and ethnicity were obtained 
from district records. 

Teacher outcome data . Teacher pretest and posttest self-report data were 
obtained with different versions of the Teacher Questionnaire, an instrument 

l0 Because the Drug Knowledge items were difficult, there must have been a 
substantial amount of guessing reducing internal consistency. 



17 



16. 



developed for this study. This instrument included measures of a) satisfac- 
tion with teaching adapted from the Purdue Teacher Morale Inventory (Rempel 
and Bentley, 1964); b) faculty cohesiveness adapted from the Teacher 
Cooperation Subscale of the Teacher Attitude and Classroom Climate Question- 
naire (Kaufman, Semmel & Agard, Note 11) and from the Intimacy Subscale of 
the Organization Climate Description Questionnaire (Halpin and Croft, 1963); 
and c) the importance and effectiveness of achieving teaching objectives 
related to the in-service training, a measure developed for this study. The 
number of items included in all pretest and posttest scales and the scales' 
internal consistency reliabilities estimated by coefficient alpha are shown 
in Table 2 for grade 7-9 teachers. 

Data Analysis 

Two approaches were employed in the analysis of student data. In the 
first approach, treatment versus control group differences were examined. 
This approach included analyses of pretest data which explored biases due to 
attrition and initial equivalence. Then posttest data were analyzed control- 
ling for some of the initial differences. The second approach examined 
students within the treatment group in order to determine whether posttest 
changes were related to the students 1 amount of exposure to ECM- trained teachers. 

A similar strategy was employed in the anlaysis of the teacher data. 
First, treatment-control differences were examined. Then, differences between 
treatment school teachers who participated in either year of ECM and control 
teachers were examined. 

In the present study, both the assignment to condition and the delivery 
of treatment was at the school level. Therefore, the appropriate unit of 
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Table 2 

teacher pretest (n = 95) and post-test (n = 76) scales, number of items, amd 
internal consistency reliabilities (coefficient alpha) 



Pretest Posttest 



Number of Number of 

Scale Items Reliability Items Rel i ability 

ECM Objectives 

. (ECM Object) 6 .70 6 .78 

Teacher Satisfaction 

(Teacher Satis) 3 .79 8 .82 

Faculty Cohesiveness 

(Faculty Cones) 7 .89 7 .£6 
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analysis is the school. However, each condition consists of only one school 
which precludes performing statistical analysis at the school level. Analysis 
of data at the student level is likely to produce spurious results due to 
both excess power and bias produced by the statistical interdependence among 
students within each school. Since these data must be analyzed at the student 
level, we have decided to compensate somewhat for the excess power by setting 
the Type I error for each analysis at .01. With this alpha level and power 
set at .80, the analysis of variance is capable of detecting a minimum effect 
size of .48 SD to .62 SD. n The analysis of covariance is capable of detecting 
a minimum effect size of .42 SD to .55 SD with a pretest posttest correlation 
of .34 (the median r). As ..any univariate analyses were conducted, isolated 
effects must be interpreted cautiously, because they may be due to Type I error. 
Hence, interpretation of results is based on patterns in the data racher than 
isolated findings. 

Rules were established for handling missing data in computing scale 
scores. A scale score was computed for a student if at least 60% of the 
items comprising that scale were completed. Any missing item score was re- 
placed by the mean for that item in the appropriate cell of the experimental 
design. This procedure utilized most of the item data and provided unbiased 
cell means. However, the procedure constrained cell variances and inflated 
degrees of freedom artificially. When more than 40% of the items comprising 
a scale were missing, the student received a missing value for that scale, 
and the case was dropped from any analysis involving that scale. 

U SD refers to the pooled wi thin-groups standard deviation. 
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Process Evaluation 

Teacher Feedback on the ECM Training Sessions 

At the end of all but the final training session, teachers anonymously 
completed a one-page "feedback form." This form solicited ratings and 
comments regarding the session. Teachers rated each session on interest, 
organization, usefulness, and enjoyableness , using five-point rating scales 
with higher numbers signifying more positive ratings. 

Teachers rated all sessions favorably. Of the 24 separate ratings 
(four per session), 20 averaged above 4.5, and none averaged less tha^ 4.4. 
The teachers frequently wrote compliments and rarely had critical comments. 

Questionnaire Surveys of Participating Teachers 

At the completion of training (January 1980) and again near the end of 
the school year (May 1980), all participating teachers in the experimental 
group completed questionnaires about ECM implementation. Teachers reported 
how often they used each in-service skill, rated the utility of each skill, 
and rated their own mastery of each skill. Table 3 summarizes the year-end 
data. The first four skills listed in Table 3 are the communication skills; 
the next six are cUcsroom management skills; the last "skill 11 — self-concept 
enhancing activi ties--is the series of classroom exercises used apart from 
the teaching of the regul^ curriculum. 

At the end of the year, more than half of the teachers reported using three 
of the four communication skills at least several times per week. On a daily 
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basis, reflecting feelings were most v/idely used, and clarifying responses 
were least used. Most of the teachers reported using three classroom manage- 
ment skills (positive nonverbal cues, positive verbal feedback, and negative 
nonverbal cues) at least several times per week, and many reported using 
these skills on a daily basis. Over half of the teachers reported using 
self-concept enhancement activities on a weekly basis. 

With regard to the utility of the skills, positive nonverbal cues and 
positive verbal feedback were considered most valuable, and tangible rein- 
forcers and time-out were considered least valuable. Thus, perceptions of 
utility were strongly related to reported use. The teachers generally rated 
their own mastery of the skills as "fair" to "good," seeing themselves least 
adept at using tangible reinforcers, negative verbal feedback, time-out, 
and self-concept enhancement activities. 

The mid-year questionnaire asked the teachers to evaluate the in-service 
course as a whole, using five-point rating scales. The teachers' ratings 
were very favorable with respect to interest (M = 4.39), organization 
(M = 4.50), usefulness (M = 4.11), and enjoyableness (M = 4.56). The 
teachers also rated the trainers highly. 

Trainer Observation s of ECM Classroom Implementation 

One of the trainers visited each classroom three or four times between 
December and April to observe the teachers applying tne ECM skills. These 
observations averaged 43 minutes. After each classroom observation, the 
trainer recorded a) approximately how frequently each skill was observed, 
and b) how well each skill was applied. 



Table 3 

TEACHERS' REPORTS ON FREQUENCY, UTILITY, AND QUALITY OF ECM SKILL IMPLEMENTATION 

AT THE END OF THE YEAR (N * 26) 



In-Service Skill 
I-Messages 

Clarifying Responses 
Reflecting Feelings 
Reflecting Content 

Positive Verbal Feedback 
Positive Nonverbal Cues 
Tangible Reinforcers 
Negative Verbal Feedback 
Negative Nonverbal Cues 
Time- Out 

Self-Concept Enhancement 



Percentage of Teachers 
Using the Skill at Least 

Several Times 
per Week 



64 
46 
58 
62 

76 
96 
22 
46 
80 
15 

52 



Every Day 
24 
19 
31 
27 

36 
60 
04 
19 
44 
08 

09 



Mean Ratings of Utility 
of the Skill 
(5=High; l=Low) 



3.92 
3.75 
3.92 
4.00 

4.36 
4.52 
2.46 
3.76 
3.88 
3.36 

4.04 



Mean Ratings of Mastery 
of the Skill* (4=Excel lent; 
l=Poor) 



2.83 
2.68 
2.92 
2.80 

2.92 
3.24 
2.37 
2.68 
3.00 
2.68 

2.74 



^Teachers who did not use a skill are not included in the mean rating for that skill, 
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Teachers used some skills much more frequently than others. Among the 
communication skills, teachers used reflecting content and clarifying 
responses rather frequently, and I-messages and reflecting feelings in- 
frequently. Among the classroom management skills, teachers only used 
positive verbal feedback frequently. Self-concept enhancing activities 
were observed during 37% of the classroom visits. Teachers applied the skills 
with roughly equal competency. The trainers' ratings of implementation quality 
were in the "B" to "B+" range for most, skills, varying between 3.50 and 4.44 on 
a five-point scale, and averaging 4.16 (SD = .27) across all skills. 

Some teachers used more of the skills than others. At the extremes, one 
teacher used seven different skills in more than half of the observed sessions; 
whereas another used only three different skills (M =4.26, SD = 1.33). The 
teachers also differed in how well they applied the skills. Here, the range 
in the trainers' average quality ratings (across skills and over classroom 
visits) was between 3.18 (i.e., "C") for one teacher and 4.60 (i.e., "B+") for 
another. The average quality rating for all teachers was 4.20 (SD = .36). 

Additional Observations of ECM Classroom Implementation 

The observations described above were done by the trainer; we now describe 
observations done by a researcher who was not involved in the training. The 
researcher also observed the frequency of teacher implementation of ECM- 
related behaviors in the classroom. However, only verbal behaviors were 
recorded; nonverbal behaviors and self -concept activities were not recorded. 
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Seven ECM- trained teachers were randomly selected for observation. 
Observational data were collected on three visits to each teacher's class- 
room. The first visit occurred early in the training, the second visit 
occurred about one month after training ended, and the final visit occurred 
approximately ten weeks after the training ended. On each v.sit, the 
researcher coded teacher behavior for 45 minutes. 12 A total of 13.5 hours 
of observational data were collected during 18 visits. 13 

Few ECM skills were observed. Reflecting Content was the only communi- 
cation skill used by a high proportion of teachers. Five teachers were 
observed using this skill, but none of them used it regularly. Clarifying 
responses were used regularly by two of the teachers; however, none of the 
other teachers was observed to use them. Only three instances of I-messages 
and reflecting feelings were observed during 18 classroom visits. 

Negative verbal feedback was the only classroom management skill used by 
most teachers, but this skill was not used regularly. Use of tangible rein- 
forcers or time-out not observed. A counterproductive behavior, teacher 
criticism, was included in the observation system. Four teachers were 
observed using this behavior irregularly. 



Observational data collected by a second researcher have not been 
reported due to problems with observer agreement. That most of the skills 
occurred infrequently made it difficult to adequately measure observer 
agreement. The data reported here are for the observer deemed most skilled 
by the training and research staffs at identifying ECM behaviors. The 
skilled observer identified fewer instances of ECM behaviors than the other 
observer because of her ability to discriminate ECM behaviors from ordinary 
teacher behaviors. 

13 0nly two visits were made to observe one teacher and one visit to 
observe another because these individuals left teaching during the school year. 
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ECM Classroom Implementation 

ECM Classroom implementation was measured by three different methods 
(teacher self-reports, trainer observations, and researcher observations). 
In general, teachers used reflecting content and did not use I-messages, 
reflecting feelings, tangible reinforcers or time-out. Inconsistent results 
were obtained for the other skills. Each method noted low amounts of imple- 
mentation, particularly the observational methods. 

Two factors contributed to the inconsistent findings. First, variation 
in frequency of occurrence of ECM behaviors was substantial. In order for 
classroom observations to provide reliable estimates of implementation, sub- 
stantially more hours of data collection would be needed. Second, different 
methods contain different biases. The teacher self-reports may have been 
exaggerated toward reporting greater implementation due to social desirability. 
Teacher self-report data also may have been inaccurate due to misuse of the 
ECM skill definitions, memory problems, and a crude response format. Finally, 
the researchers' observations were more conservative than the trainer's 
because the researcher reordered only ideal ECM behaviors, whereas the trainer 
recorded less than perfect behaviors and then assigned them quality ratings. 
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Results 

Initial Equivalence—Teachers 

Analyses of pretest and background measures were conducted to deter- 
mine whether a) teachers in the experimental and control conditions were 
equivalent and, b) teachers in the experimental school who participated in 
the training were similar to nonparticipating teachers. 

Experimental and control teachers had similar amounts of teaching 
experience. However, in the three years prior to this study, fewer experi- 
mental teachers than control teachers had in-service training in building 
students' self-esteem (10% vs. 44%), Schools Without Failure (47% vs. 82%), 
and Values Clarification (14% vs. 41%). Experimental teachers were initially 
lower on Faculty Cohesiveness, F (1 ,50) = 17.94, £< .001, but were similar 
to control teachers on Teacher Satisfaction, F (1,50) = 3.37, n^, and ECM 
Objectives, F (1,50) = 1.54, n^. 

Experimental participants were similar to experimental nonpartici pants 
with one exception. They were more likely to have had prior in-service 
training in Schools Without Failure (68% vs. 18%). 

Analysis of Teacher Outcomes 

Experimental and control teachers were compared on the three posttest 
outcome measures controlling for their corresponding pretest scores. Analyses 
of covariance revealed no treatment condition differences on ECM Objectives, 
F (1,64) < 1, Faculty Cohesiveness, F (1,66) < 1 , or Teacher Satisfaction, 
F (1,66) = 1.51, n*. 
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Similar analyses compared teachers in the experimental school who 
completed the training with control teachers. Again, no differences were 
found on ECM Objectives, F (1,47) < 1 , or on Faculty Cohesi veness , 
F (1,49) <1. A significant difference was found on Teacher Satisfaction, 
F (1,49) = 4.32, £ <.05, with participants scoring .55 5D higher than 
control teachers. 

Initial Equivalence and Attrition— Students 

Initial nonequi valence between conditions affects both the justification 
for attributing posttest differences to the treatment (internal validity) 
and the justification for generalizing treatment effects to the student 
population at pretest (external validity). Attrition affects internal 
validity if students missing from one condition are systematically different 
from those missing from another condition. Attrition also affects external 
validity if the attrited students are systematically different from those 
who remain in the sample. 

Analyses were conducted to determine whether a) students in the experi- 
mental and control conditions were equivalent on the pretest measures and 
b) the experimental and control groups were affected differentially by 
attrition. The means and standard deviations for all student pretest and 
posttest measures are summarized in Tables 4 and 5 by sex and treatment 
condition for grades 8-9. The final posttest results will be discussed later. 

As a test for initial equivalence between conditions, two-way analyses 
of variance were performed for each grade-sex group on each of the student 
pretest measures, with treatment condition (experimental vs. control) and 
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Table l \ 



MEANS AND STANDARD DEVIATIONS FOR STUDFNT DATA 
BY SEX BY TREATMENT CONDITION FOR GRADE 8 



Male Students Female Students 







Experimental 


Control 


Experimental 


Control 




M 




SD 


M 


SD 


M 




SD 


M 


SD 


Control Succ 


Pre 


1.82 




.17 


1.80 


.16 


1.83 




.15 


• 

1.85 


.12 




Post 1 


1 ■ Oj 




. c \ 


1 P£ 

1 .oO 


1 Q 


1 .OD 




.17 


1 .90 


.14 




Pnct 9 
r u o L c 


1 .78 




.19 


1.76 


.19 


1.82 




.15 


1.83 


.19 


Control Fail 


Pre 


1 .72 




.20 


1.63 


.24 


1.70 




.19 


1 .67 


.21 




Post 1 


1 7"? 






1 . / D 


99 


1 7 C 


• 


. c 1 


1 .82 


.19 






1 .70 




.24 


1.68 


.24 


1.74 




.19 


1.74 


" .24 


Acad Self 


Pre 


1 83 
I .oo 




99 


1 .oD 


on 
. cv 


1 .OJ 




. 21 


1 .82 


.24 




Pn«;t 1 


2.41 




.39 


2.45 


.42 


2.35 




.41 


2.48 


.39 




Post 2 


2.74 




.45 


2.81 


.52 


2.76 




.54 


2.81 


.54 


Social Self 


Pre 


i an 

1 .ou 




91 


1 7£ 


OF 

. do 


1 .o 1 




.21 


1 .82 


.19 






1.79 




.25 


1.79 


.23 


1.82 




.23 


1.89 


.14 




Post 2 


2.95 




.44 


3.08 


.32 


3.10 




.42 


3.17 


.38 


Affec Climate 


Pre 


1 .82 




.24 


1.82 


.19 


1.88 




.19 


1.84 


.18 




Post 1 


2.28 




.36 


2.38 


.31 


2.34 




.38 


2.46 


.36 




Post 2 


2.60 




.50 


2.72 


:s3 


2.76 




.48 


2.79 


.57 


Att Sch 


Pre 


1 .68 




.26 


1.74 


.25 


1.83 




.22 


1 .86 


.17 




Post 1 


1.60 




.31 


1.68 


.29 


1.70 




.31 


1.78 


.23 




Post 2 


2.32 




.64 


2.46 


.76 


2.66 




.70 


2.79 


.63 


Peer Att Sch 


Post 1 


2.59 




.38 


2.71 


.42 


2.67 




.42 


2.75 


.35 




Post 2 


2.40 




.41 


2.52 


.44 


2.54 




.39 


2.45 


.38 


GPA 


Pre 


3.59 




.57 


3.73 


.47 


3.70 




.53 


3.82 


.54 




Post 1 


3.49 




.56 


3.75 


.53 


3.42 




.57 


3.70 


.59 




Post 2 


3.27 




.75 


3.59 


.76 


3.51 




.87 


3.73 


.62 


Unex Abs 


Post 1 


1 .05 


2 


.16 


1.82 


2.11 


1.06 


1 


.69 


1 .24 


1.99 




Post 2 


1.51 


2 


.13 


1.64 


2.23 


2.10 


5 


.52 


1 .35 


1.70 


Non-drug Prob 


Post 1 


1.23 


2 


.15 


.30 


.86 


.44 


1 


02 


.37 


1.12 




Post 2 


1.31 


2 


.50 


.68 


1 .66 


.71 


1 


51 


.26 


.71 


Drug Prob 


Post 1 


.02 




13 


.00 


.00 


.01 




10 


.04 


.19 




Post 2 


.35 




23 


.00 


.00 


.07 




25 


.09 


.35 
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Table 4 (continued) 



Male Students Female Students 



Measure 


Test 


Experimental 
M SD 


Control 
M SD 


Experimental 
M SD 


Control 
M SD 


Nnowieage 


Post 1 
Post 2 


3.25 1.41 
2.57 1.28 


3.06 1.46 
2.45 1.17 


2.93 1.46 
2.77 1.21 


2.44 
2.33 


1 .13 
1 OR 


General Drug Att 


Pre 
Post 1 
Post 2 


1.89 .63 
1.98 .78 
2.32 .93 


1.92 .64 
2.04 .68 
2.32 .94 


1.86 .59 

2.10 .88 
2.26 .86 


1 .98 
2.23 
2 21 


.73 
.86 
■ y \ 


Soft Att 


Pre 
Post 1 
Post 2 


1.72 .66 
1.91 .79 
2.24 .89 


1.70 .64 
1.89 .70 
2.24 1,00 


1.75 .73 
2.13 .85 
2.21 .92 


1 .83 
2.23 
2 18 

Cm m 1 U 


.77 
1.04 
96 


Hard Att 


Pre 
Post 1 
Post 2 


1.19 .37 

1 in c i 
\ .60 . b 1 

1.52 .92 


1.18 .38 
1.26 .43 
1.49 .85 


1.25 .49 
1.46 .76 
1.37 .55 


1.22 
1.42 
1 .33 


.50 
.66 
.61 


Soft Peer Att 


Pre 
Post 1 
Post 2 


2.15 .99 
2.51 1.05 
2.89 1.12 


2.13 .86 
2.33 1.05 
2.67 1.09 


2.25 .97 
2.78 1.07 
3.02 1.02 


2.25 
3.01 

O . CO 


.96 
1 .15 

1 . 1 D 


Hard Peer Att 


Pre 
Post 1 
Post 2 


1.55 .81 
1.80 .99 
2.03 1.16 


1.48 .68 
1.57 .83 
1.92 .98 


1.64 .80 
1.94 1.00 
2.09 .90 


1.47 
2.02 

L . 1 J 


.60 
1.11 

1 . U 1 


Ale Benefits 


Pre 
Post 1 
Post 2 


1.57 .61 

1.58 .69 
1.79 .77 


1.50 .50 
1.56 .59 
1.80 .88 


1.54 .60 
1.66 .69 
1.70 .71 


1.49 
1 .67 
1 .76 


.63 
.68 
79 


Pot Benefits 


Pre 
Post 1 
Post 2 


1.59 .67 

1 CO "7 *3 

1 .bo .73 
1.83 .94 


1.47 .52 
1.63 .72 
1.88 .94 


1.50 .64 
1.69 .80 
1.68 .77 


1.54 
1 .67 
1.74 


.77 
.75 
.88 


Dill DanA^ifr 


Pre 
Post 1 
Post 2 


1.53 .61 
1.48 .72 
1.47 .80 


1.39 .52 
1.34 .51 
1.49 .75 


1.40 .54 
1.51 .63 
1.46 .60 


\ • oc. 

1.37 
1 .52 


. 0 1 

.50 
.79 


Ale Costs 


Post 1 
Post 2 


1.92 .71 
2.11 .70 


1.88 .62 
2.14 .82 


1.98 .71 
2.01 .67 


1.98 
2.00 


.70 
.73 


Pot Costs 


Post 1 
Post 2 


1.72 .71 
2.02 .86 


1.79 .77 
1.94 .89 


1.87 .83 
1.97 .77 


1.95 
1.92 


.79 
.85 


Pill Costs 


Post 1 
Post 2 


1.57 .67 
1.70 .83 


1.50 .58 
1.64 .77 


1.72 .73 
1.75 .73 


1 .60 
1.56 


.75 
.64 


Soft Peer Use 


Post 1 
Post 2 


28.66 21.48 
33.30 21.48 


21.72 17.03 
33.68 19.62 


30.01 17.28 
36.36 19.18 


34.34 
44.52 


24.00 
22.33 


Hard Peer Use 


Post 1 
Post 2 


8.94 13.21 
9.13 16.10 


4.46 7.86 
10.08 12.81 


11.59 13.37 
8.48 13.40 


9.01 
9.56 


13.10 
12.20 



Table 4 (continued) 



Male Students 







Experimental 


Control 


Measure 


Test 


M 


SD 


M 


SD 


Ale Involve 


Pre 


2 09 


1 15 


? ?4 

Cm • L.*T 


l . C J 




Post 1 


2.16 


.94 


2.27 


1.05 




Post 2 


2.66 


1.08 


2.48 


1.12 


Cig Involve 


Pre 


i zip 

1 . to 


Q7 


1 G9 


i in 
1 . 1 U 




Post 1 


1.50 


.71 


1 .62 


1.00 




Post 2 


1 .77 


1.00 


1 .86 


1.18 


Pot Involve 


Pre 


1 . 1 C 


. HO 


1 .0/ 


.y i 




Post 1 


1.34 


.84 


1.57 


.98 




Post 2 


1.69 


1.10 


1.96 


1.35 


Inhalant Involve 


Pre 


1 91 


C7 
.0/ 


1 .CO 


.71 






1.08 


.33 


1.07 


.20 




Post 2 


1.24 


.71 


1.21 


.57 


Barbiturate Involve 


Pre 


I .Ub 




i n/i 
1 .U4 


. lo 






1.04 


.17 


1.04 


.17 




Post 2 


1 .17 


.59 


1.18 


.54 


Amphetamine Involve 


Pre 


1 . \jC 


. 1 1 


1 .Ub 


1 9 




Prist 1 


1.04 


.20 


1.04 


.17 




Post 2 


1.18 


.57 


1.23 


.67 


Cocaine Involve 


Pre 


1 .uo 


90 
. OJ 


1 .Ub 


9*2 

. do 




Pnct 1 


1.04 


.15 


1.00 


.01 




Post 2 


1.26 


.69 


1.17 


.50 


PCP Involve 


Pre 


i nn 


. Uj 


i n/i 

1 . U*f 


IP 

. lo 




Prist 1 


1.01 


.08 


1.00 


.01 




Post 2 


1.15 


.57 


1 .10 


.44 


LSD Involve 


Pre 


1.01 


.05 


1 .00 


.00 




Post 1 


1.01 


.07 


1 .03 


.13 




Post 2 


1.10 


.39 


1.14 


.64 


Heroin Involve 


Pre 


1.02 


.09 


1.02 


.11 




Post 1 


1.01 


.13 


1.00 


.00 




Post 2 


1.16 


.67 


1 .16 


.63 



Female Students 



Experimental Control 



M 


SD 


M 


SD 


1 • OO 


■ JO 


c . u/ 


1 


2.31 


.97 


2.32 


1.00 


2.47 


1.09 


2.33 


1.07 


1 . DO 


Q7 


1 70 

1 ./o 


i 0 0 
1 ,6c 


1 .97 


1 .22 


2.17 


1 .27 


2.12 


1.33 


2.38 


1.52 


i in 
1 . IU 


AO 


1 "31 

1 .0 1 


7 A 


1 .45 


.97 


1.61 


1.04 


1.65 


1.03 


1.77 


1.22 


i no 


.oU 


1 17 
1.1/ 


■7 n 
.70 


1.12 


.35 


1.13 


.44 


1.11 


.39 


1.11 


.42 


i m 
1 .U 1 


no 


i no 
1 .03 


.16 


1 .06 


.22 


1.11 


.32 


1.07 


.33 


1.06 


.24 


i m 
1 .U 1 


nc 
.U!i 


i no 
1 .Uo 


. 1 7 


1.09 


.33 


1.07 


.23 


1.16 


.58 


1.12 


.45 


i no 
1 .vd 


. 1 / 


1 A1 

1 .01 


.05 


1.09 


.32 


1.08 


.25 


1.14 


.50 


1 .22 


.75 


1 . U 1 


. 1 1 


i n 1 
1 .U 1 


nc 
.05 


1.08 


.40 


1.02 


.10 


1.01 


.08 


1.04 


.27 


1.00 


.00 


1.00 


.00 


1.07 


.34 


1.03 


.14 


1.01 


.11 


1.10 


.51 


1.04 


.35 


1.03 


.16 


1.06 


.30 


1.02 


.10 


1.01 


.14 


1 .03 


.14 
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Table 5 



MEANS AND STANDARD DEVIATIONS FOR STUDENT DATA 
BY SEX BY TREATMENT CONDITION FOR GRADE 9 









Male Students 






Female 


Students 




Measure 




Experimental 


Control 


Experimental 


Control 


Test 


M 


SD 


M 


SD 


M 


SD 


M 


SD 


Control Succ 


Pre 


1.78 


.18 


1.81 


.16 


1.80 


.17 


1.85 


.14 




Pnst 1 


1 • oo 


. 1 O 


1 .OC 


. lo 


1 Q7 

1 .o7 


.17 


1 .85 


.18 




Post 2 


1.86 


.15 


1 .78 


.20 


1 .84 


.15 


1 .84 


.18 


Control Fail 


Pre 


1 .64 


.23 


1.68 


.24 


1 .65 


.21 


1.70 


.23 




Pn<;t 1 


1 77 


. CO 


1 £7 
1 .0/ 


. CO 


1 77 
1 .// 


. iy 


1 .75 


.22 




Pn<;t ? 


1.75 


.21 


1.67 


.26 


1.78 


.18 


1.77 


.22 


Acad Self 


Pre 


1 77 


. CO 


1 .0 1 


OA 
. CH 


1 .lb 


. do 


1 .86 


.20 




PnQt 1 

rUi L 1 


2.34 


.36 


2.41 


.36 


2.32 


.44 


2.54 


.43 




Post 2 


2.76 


.44 


2.75 


.46 


2.74 


.51 


3.00 


.49 




Prp 

r i C 


1 ft? 
1 .OC 


. lo 


1 - O 1 


. lo 


1 .0/ 


. 1 0 


1 .90 


.13 






1 .82 


.18 


1.84 


.19 


1 .89 


.16 


1 .90 


.15 




Post 2 


3.03 


.33 


3.00 


.40 


3.13 


.30 


3.21 


.33 


Ml 1 CL V* 1 IMIu UC 


Pro 


1.73 


.28 


1 .84 


.22 


1 .77 


.27 


1 .91 


.18 




Post 1 


2.24 


.32 


2.25 


.34 


2.25 


.39 


2.38 


.39 




Post 2 


2.78 


.39 


2.62 


.49 


2.77 


.51 


2.92 


.48 


Aft ^rh 


Dmq 

ric 


1.58 


.28 


1.61 


.28 


1 .74 


.26 


1 .82 


.23 




Post 1 


1.53 


.27 


1.56 


.28 


1 .63 


.31 


1 .70 


.30 




Post 2 


2.43 


.53 


2.44 


.67 


2.58 


.63 


2.70 


.68 


Doov* A +• t- C #~h 

rcci mll ocn 


rOSt 1 


2.53 


.36 


2.58 


.40 


2.53 


.41 


2.65 


.37 






2.54 


.32 


2.43 


.37 


2.50 


.41 


2.48 


.43 


GPA 


Pre 


3.35 


.64 


3.50 


.58 


3.49 


.54 


3.72 


.59 




Post 1 


3.37 


.68 


3.45 


.63 


3.47 


.57 


3.73 


.64 




Post 2 


3.39 


.70 


3.61 


.78 


3.52 


.67 


3.95 


.77 


Unex Abs 


Post 1 


1.71 


2.57 


1.56 2 


.56 


1.51 


2.19 


1.78 


3.61 




Post 2 


2.19 


2.83 


2.44 3 


.44 


2.31 


3.86 


1.71 


2.66 


Non-drug Prob 


Pre 


.68 


1.42 


.28 


.67 


.13 


.53 


.09 


.37 




Post 1 


1.29 


2.50 


.49 


.8(. 


.73 


1.86 


.24 


.72 




Post 2 


1.27 


2.50 


.32 


.68 


.54 


1.46 


.19 


.52 


Drug Prob 


Pre 


.00 


.00 


04 


.26 


.01 


.09 


.00 


.00 




Post 1 


.01 


.09 


.01 


.12 


.04 


.33 


.01 


.11 




Post 2 


.05 


.21 


.04 


.26 


.04 


.24 


.02 


.15 



ERIC 



33 



Table 5 (continued) 



Male Students 









Experimental 


Control 


Measure 


Test 




M 


SO 


M 


SO 


Know! edae 




1 


3.38 


1.33 


3.07 


1.34 




Post 


2 


2.90 


1.29 


2.74 


1.42 


General Drug Att 


Pre 




9 IP. 


. / o 


9 ^fi 


.0 1 






1 

I 


2.24 


.77 


2.48 


.94 




Post 


2 


2.42 


.83 


2.65 


.89 


Soft Att 


Pre 




9 ni 


ft! 
.0 1 


9 HQ 
C . uy 


7Q 

. /y 




Post 




2.15 


.79 


2.48 


.91 




Post 


2 


2.27 


.80 


2.57 


.87 


Hard Att 


Pre 




1 .JJ 


. DD 


1 /IT 
1.41 


£9 
. OC 




Prist 


1 


1.37 


.49 


1.63 


.76 




Post 


2 


1.39 


.56 


1.86 


1.02 


Soft Peer Att 


Pre 




<:. /o 


. y/ 


9 OO 

c.oc 


1 .Uo 




Prist 


1 


3.09 


.90 


3.11 


1.13 




Post 


2 


3.09 


.96 


3.10 


1.00 


Hard Peer Att 


Pre 




1 . /o 


.91 


1 .96 


.yy 




Prist 


1 


2.01 


1.03 


2.43 


1.24 




Post 


2 


2.00 


.90 


2.16 


1.06 


Ale Benefits 


Pre 




1 . / 3 


cc 
.00 


1 .75 


7-7 




Prist 


1 


1.84 


.72 


1.94 


.80 




Post 


2 


1.92 


.72 


2.07 


.80 


Pot Benefits 


Pre 




1 Q7 
1 .0/ 


.Ob 


1 Q9 
1 .OO 






Prist 


1 


1.92 


.89 


2.10 


.95 




Post 


2 


1.89 


.87 


2.12 


1.01 


Pill Bpnpfits 

• III UCIICI 1 


Pr p 




1.61 


.73 


1.61 


.72 




Post 


1 


1.54 


.67 


1.62 


.79 




Post 


2 


1 . HO 


57 


1 5.5 




Ale Costs 


Post 


1 


2.11 


.69 


2.15 


.75 




Post 


2 


c . cc 


fi7 


9 




Pot Costs 


Post 


1 


2.07 


.78 


2.20 


.85 




Post 


2 


9 19 


O A 

.84 


f . oU 


.O 1 


Pill Costs 


Post 


1 


1.68 


.65 


1.75 


.75 




Post 


2 


1.66 


.65 


1.87 


.79 


Soft Peer Use 


Post 


1 


38.68 


19.16 


41.61 


20.68 




Post 


2 


42.85 


19.30 


42.09 


16.44 


Hard Peer Use 


Post 


1 


8.34 


12.75 


16.10 


15.84 




Post 


2 


8.71 


15.91 


10.25 


9.39 



Female Students 



Experimental Co ntrol 



M 


SO 


M 


SD 


3.20 


1.23 


3.34 


1.42 


2.98 


1 .32 


2.80 


1.37 




P.7 


9 f)5 

C ,UJ 


P. "3 


2.50 


.79 


2.24 


.87 


2.52 


.84 


2.35 


.92 


2 29 


95 


1 Q.P. 
1 . 00 




2.45 


.76 


2.24 


.88 


2.58 


.84 


2.45 


.97 


1 53 


19 

■ / C 


1 ?4 


• HO 


1.55 


.64 


1.46 


.66 


1.59 


.67 


1.51 


.76 


7 9Q 


i no 

1 .Uo 


Q 1 Q 
O . 1 O 


1 1 0 

1 . iy 


3.46 


.81 


3.10 


1 .09 


3.47 


.77 


3.18 


1 .02 


9 QQ 

£ . oy 


1 .UO 




1 .Uo 


2.38 


.92 


2.18 


1 .04 


2.24 


.91 


2.25 


1.05 


1 .OH 


.DO 


1 . t>y 


CO 

.00 


1.88 


.74 


1.70 


.69 


2.00 


.71 


1.73 


.75 


( .Oil 


. yu 


1 7 A 


. y 1 


1.97 


.81 


1.82 


.80 


2.07 


.86 


1.84 


.86 


1.56 


.70 


1.55 


.74 


1.70 


.76 


1 .54 


.63 


1 70 

1 . / V 


78 


1 51 


70 


2.35 


.62 


2.02 


.64 


2.38 


.66 


1 99 


75 


2.27 


.76 


1.99 


.78 


2.37 


.77 


9 fi£ 
C ♦ UD 


.OO 


1.90 


.72 


1.75 


.70 


1.93 


.72 


1.70 


.73 


48.08 21 .08 


46.95 


22.88 


50.34 


19.41 


49.13 


19.50 


14.67 


15.55 


13.80 


15.06 


11.39 


13.62 


12.16 


10.79 
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Table 5 (continued) 



-Male Students 



Measure 




Experimental 


Control 


Test 


M 


SO 


M 


SO 


Ale Involve 


Pre 






9 19 






r vol i 


2.41 


.95 


2.63 


1.05 




Post 2 


2.77 


1 .03 


2.94 


1.07 


Cig Involve 


Pre 


1 . oo 




i P.P. 

1 .oo 


1 99 




Po<;t 1 


1.68 


.96 


1.80 


1.00 




Post 2 


1.76 


1 .06 


2.05 


1.34 


Pot Involve 


Pre 


1 . bl) 


1 i c 
1 . lb 




i /it 
1.41 




PfKt 1 
rud l i 


1.75 


1 .06 


2.01 


1.32 




Post 2 


2.01 


1.21 


2.39 


1.46 


Inhalant Involve 


Pre 


i no 

i . uy 


OQ 
. CO 


i on 
1 . oil 


.Id. 




Pnc * 1 


1.08 


.27 


1 .31 


.65 




Post 2 


1.15 


.43 


1 .33 


.75 


Barbiturate Involve 


Pre 


1 . u/ 


oo 

• £0 


1 c6 


C7 






1.11 


.38 


1.21 


.61 




Post 2 


1.15 


.45 


1 .27 


.63 


Amphetamine Involve 


Pre 


i .uy 


A O 


1.10 


OO 

.do 




rub L 1 


1.06 


• 27 


1.28 


.71 




Post 2 


1.20 


.58 


1.36 


.72 


Cocaine Involve 


Pre 


i nd 


ifi 


1 17 
1.1/ 








1.08 


.32 


1.28 


.79 




Post 2 


1.21 


.61 


1 .42 


.78 


PCP Involve 


Pre 


1 .U6 


. 13 


1 .06 


OO 






1.03 


.25 


1 .12 


.42 




Post 2 


1.07 


.42 


1.12 


.34 


LSD Involve 


Pre 


1.02 


.10 


1 .05 


.26 




Post 1 


1.06 


.28 


1 .04 


.26 




Post 2 


1.04 


.21 


1.19 


.50 


Heroin Involve 


Pre 


1.02 


.08 


1 .02 


.10 




Post 1 


1.04 


.29 


1.10 


.36 




Post 2 


1.03 


.19 


1 ,11 


.33 



■Female Students 



Experimental Control 



M 




SO 


M 


SO 


9 57 


1 


45 


9 3fi 


1 . 0 1 


2.68 




.98 


2.26 


1 .06 


2.98 


1 


.01 


2.48 


1.03 


9 3K 


1 
1 


79 


9 01 ' 


1 

1 . DO 


2.29 


1 


.37 


2.17 


1.36 


2.51 


1 


.36 


2.35 


1.44 


1 . b 1 


1 


.Ul 


1 .51 


1 .06 


1 .82 


1 


.19 


1.73 


1.10 


2.16 


1 


.26 


2:02 


1.31 


1 . C 1 




. DU 


1 H7 
1 , U/ 


on 
.01) 


1.17 




.42 


1.07 


.23 


1.10 




.30 


1.11 


-.38. 


1 Ol 




• 57 


l.ll 


o a 


1.19 




.40 


1.04 


.25 


1.23 




.61 


1.14 


.48 


1 . Id 




£ 9 
.00 


1 in 
1 . 1 0 


C 1 

. b 1 


1.22 




.49 


1.15 


.44 


1.41 




.77 


1.30 


.76 






• 


i 


1 Q 


1 .17 




.47 


1 .08 


.30 


1 .20 




.45 


1.28 


.70 


1 .(Jo 




oo 
.66 


1 .11 1 


.0b 


1.10 




.44 


1.03 


.14 


1.06 




.27 


1 .05 


.29 


1.03 




.18 


1.04 


.18 


1 .08 




.30 


1 .03 


.22 


1.09 




.31 


1.10 


.38 


1.04 




.17 


1.01 


.11 


1.05 




.26 


1.01 


.05 


1.04 




.36 


1.04 


.17 
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33. 



attrition status (attrited vs. non-attrited) as the factors. A ma^'n effect 
for treatment condition would suggest initial nonequivalence, and hence 
limit both types of validity. A main effect for attrition status would 
limit external validity. The interaction between treatment condition and 
attrition status would limit internal validity. 

For grade 8 males no significant treatment x attrition interaction or 
treatment main effect was obtained. Significant attrition status main effects 
were obtained on four measures. As compared to non-attrited students, attrited 
students had lower GPA, and were higher on General Drug Att, Soft Att, and 
Pot Involve. 

For grade 8 females no significant treatment x attrition interaction cr 
treatment main effect was obtained. Significant attrition status main effects 
were obtained on seven measures. As compared to non-attrited students, 
attrited students were lower on Control Succ, and higher on Pill Benefits, 
Pot, Barb, Coc PCP and LSD Involve. 

For grade 9 males no significant treatment x attrition was obtained, 
but one treatment main effect obtained significance. As compared to their 
controls, the experimental students were lower on Affec Climate. Significant 
attrition status main effects were obta+ned on 13 measures. As compared to 
non-attrited students, attrited students were lower on Acad Self, Social Self, 
Affec Climate, and Att Sch, and were higher on 13 of the 18 drug-related 
measures . 

For grade 9 females no treatment x attrition interaction obtained signi- 
ficance, but two treatment main effects did. As compared to controls, 
experimental students were lower on Affec Climate and higher on Hard Att. 
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Two attrition status main effects obtained significance. As compared to 
non-attrited students, attrited students were lower on Att Sch and higher 
on Pot Involve. 

In sum, there was substantial evidence for initial equivalence of 
treatment and control students. However, the teaching climate was perceived 
more positively in the control school by the 9th graders. A consistent 
pattern of effects for attrition status indicated that attrited students 
were less pro-school and more pro-drug than students who remained in the 
analysis. These results limit the general izability of our findings. 

Analysis of Student Outcomes 

In order to attribute posttest differences between conditions to the 
treatment, initial differences between treatment conditions must be statisti- 
cally controlled. Ideally, each posttest measure should be adjusted for all 
related pre-existing differences and the pretest measures should be error 
free. The analysis of covariance using the corresponding pretest as a 
covariate is a common technique that approximately controls for pre-existing 
differences; however, with a nonequivalent control group design this technique 
does not assure proper adjustment for initial differences (Reichardt, 1979). 

Due to the heterogeneity of variance and covariance in most of the 
measures, separate analyses of covariance were performed for each grade-sex 
combination. Although this approach substantially increases the number of 
statistical tests performed (by a factor of four) and hence the chance of 
Type I error, it is less likely to violate the assumptions underlying the 
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analysis. In addition, it obviates the need to conduct a posteriori tests 
of simple effects. The large number of tests conducted (140) requires a 
search for meaningful patterns. 

A one-way (treatment condition) analysis of covariance was conducted 
on each posttest measure with the corresponding pretest as the covariate. 15 
Exceptions to this procedure were made for those variables not measured at 
pretest: Att Sch was used as a covariate for Peer Att Sch; GPA for Unex 
Abs; Soft Att for Ale Costs, Pot Costs and Pill Costs; Soft Peer Att for 
Soft Peer Use; Hard Peer Att for Hard Peer Use; Pot Involve for Knowledge and 
Drug Problems. 16 In addition, for the grade 8 analyses, GPA was used as a 
covariate for Non-drug Problems. 

Table 6 summarizes the results of these ANCOVAS. No significant treat- 
ment effects were obtained for grade 8 males or for grade 8 females. Positive 
treatment effects were found for grade 9 males on Affec Climate, Control Succ, 
Control Fail, and Hard Att. For grade 9 females, no positive treatment 
effects were found, and negative effects were obtained on GPA, Ale Benefits, 
Ale Costs, and Ale Involve. 

The assumptions for the analysis of covariance specify that the within- 
group regression coefficients are homogeneous, that the treatment and regression 
effects are additive and related linearly to the dependent variable, and that 
the error term is normally distributed with a mean of zero and the same 
variance for each group. 

1 Additional analyses of covariance employed the corresponding pretest 
and the pretest- treatment interaction as covariates. The latter term allows 
for different regression slopes in the two conditions. In 14 of the 140 analvses 
(3%), the precest-treatment interaction term was significant. These results did 
not form any consistent pattern, nor did they substantially affect the interpre- 
tation of the results from the ANCOVA employing only the pretest as a covariate. 

1 6 

Pretest Drug Problems was not used in this or any other analysis because 
it had little variance and did not correlate with posttest Drug Problems. 
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Table 6 

summary of analyses of covariance on student data 



Measure 


Grade 8 
Male 


Grade 8 
Female 


Grade 9 
Male 


Grade 9 
Femal e 


Affec Climate 


1 .42 


<1 


13.58 + 


<1 


Social Self 


5.25 


1.34 


<1 


1.49 


Acad Self 


<1 


1 .13 


<1 


3.30 


Control Succ 


<1 


<1 


15.52 + 


<1 


Control Fail 


<1 


<1 


8.12 + 


1 .72 


Att Sch 


<1 


<1 


<1 


<1 


Peer Att Sch 


1.57 


4.84 


6.54 


1 .61 


6 PA 


4.11 


1 .48 


1.84 


10.80" 


Unex Abs 


<1 


<1 


<1 


1 .54 


Non-drug Prob 


1.40 


3.34 


6.26 


4.25 


Drug Prob 


3.00 


<1 


<1 


1 .00 


General Drug Att 


<1 


<1 


<1 


1 .01 


Ale Benefits 


<1 


<1 


1.66 


9.07" 


Pot Benefits 


<1 


<1 


3.87 


6.00 


Ale Costs 


<1 


<1 


1.12 




Pot Costs 


3.28 


<1 


1.13 


3.34 


Soft Att 


1.12 


<1 


4.04 


<1 


Knowl edge 


<1 


4.87 


<1 


<1 


Soft Peer Att 


1.58 


<1 


<1 


2.03 


Soft Peer Use 


<1 


5.52 


<1 


<1 


Ale Involve 


3.22 


1.97 


<1 


9.57" 



ERIC 



3y 



Table 6 (continued) 
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Measure 


Grade 8 

rid 1 c 


Grade 8 

Pom a 1 a 
rciud 1 c 


Grade 9 
na I e 


Grade 9 
remale 


Cio Involve 

w i jj illy v l Y v 


^ 1 


^ 1 


9 19 


<1 


Pot Tnvolvp 


< 1 


< 1 


i on 


1 .75 


Pill Benefits 


< 1 


1 .00 


O 09 
0 .CC 


3 •70 


Pill Costs 


9 09 


9 /II 


A 70 

4.73 


2.59 


Hard pppr Atf* 
naru rccr nil 


l /In 


< 1 


<1 


<1 


Hard Pppk* IIcp 
naru rccr UoC 


< 1 


< 1 


1 .05 


<1 


Hard Att 


9 1 "3 
£ . 1 0 


< 1 


1^.39 


<1 


Inhalant Tnv/nlv/P 


< 1 


< 1 


<1 


<1 


Barhituratp Tnvolvp 

UU 1 U 1 LU 1 U 1 II V V 1 VC 


< 1 


< 1 


1 /I Q 


1 /IT 

1 .41 


AmDhptami np Tnvolvp 


>> 1 




< 1 


d.00 


Cocaine Involve 


1 .61 


1.78 


1 .62 


<1 


PCP Involve 


1 .01 


<1 


3.08 


<1 


LSD Involve 


<1 


3.10 


5.50 


<1 


Heroin Involve 


<1 


2.92 


4.01 


<1 



NOTE: The values tabled are the F-ratios for the treatment condition effect. 

The degrees of freedom for the error term are 126-163 for grade 8 males, 
119-156 for grade 8 females, 151-201 for grade 9 males, 148-200 for 
grade 9 females. 

" = Significant negative effect (p_ < .01) 

+ = Significant positive effect (£ < .01) 
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The results showed no obvious pattern. There were no significant treat- 
ment differences in 132 of the 140 analyses, four positive treatment effects, 
and four negative effects. The positive effects v/ere confined to grade 9 
males, and the negative effects were confined to grade 9 females. The 
expected pattern, that positive treatment effects would occur primarily for 
school-related measures, was obtained only for the grade 9 males. 

Relation of Student Outcomes to Treatment Exposure 

Student data from the experimental school were analyzed to determine 
whether student posttest changes were related to amount of exposure to 
ECM-trained teachers (Treatment Expos). We assumed that a monotonic function 
describes the relationship between Treatment Expos and the pretest-adjusted 
posttest measures. 

Treatment Expos was trichotomized. Students in the bottom third of the 
distribution had 0-6 ECM-trained teachers (K = 4.68), the middle third had 
7-9 ECM teachers (M = 8.01), and the upper third had 10-16 ECK teachers 
(M = 11.27). The trichotomized Treatment Expos was coded into two orthogonal 
components: linear and quadratic. Each posttest measure was regressed in a 
step-wise procedure on its corresponding pretest and on the two Treatment r xpr>s 
components. These analyses were performed separately on each grade-sex group 
of students in the treatment school. 

Either a significant linear Treatment Expos component by itself or 
significant linear and quadratic components together could be interpreted as 
a treatment effect. If the quadratic component were significant and the linear 
component, were not, a treatment effect would not be implied because the rela- 
tion would not be monotonic, 

ERIC 41 



In none of the 140 regressions was the linear component of Treatment 
Expos significant. In one instance the quadratic component was significant. 
Thus, there was no evidence that student posttest changes were monotonically 
related to the amount of exposure to ECM- trained teachers. 
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Discussion 

One of the three hypothesized effects of ECM on teacher outcomes was 
obtained. The predicted positive effect on satisfaction with teaching was 
significant for participants but not for all experimental teachers. Effects 
were not obtained on faculty cohesiveness or effectiveness at achieving ECM 
objectives. 

ECM did not appear to affect student outcomes. No hypothesized effects 
were found for 8th grade students or 9th grade females. Hypothesized 
effects were obtained for 9th grade males on perceptions of the teaching 
climate and locus of control; however, these effects did not form an inter- 
pretable pattern. Furthermore, experimental student outcomes were not 
systematically related to exposure to ECM-trained teachers. 

The present study employed a nonequivalent control group design; however, 
there was little evidence for initial nonequivalence. It seems unlikely that 
the lack of positive effects was due simply to a weak design. More plausible 
explanations might be failure of teachers to implement the ECM strategy or 
failure of the ECM strategy itself. To the extent that our process evaluation 
supports implementation failure, the effectiveness of the strategy itself 
remains an open question. Nevertheless, implementation levels are an index 
of the practical utility of the program, and as such, are informative. 

Classroom implementation of the ECM skills in both the present and 
previous study (Schaps et al., Note 4) was highly disappointing. This was 
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true despite several changes in the training for the second year. It 
appears undeniable that the substance of ECM was either too subtle, not 
useful, or difficult to put into practice in the classroom (see Moskowitz, 
Malvin, Schaeffer, Schaps and Condon for a discussion of classroom oppor- 
tunities for skill application, Note 12). Therefore, ECM may not be an 
effective prevention strategy unless means are devised to improve teacher 
implementation. Even so, such means are likely to be extraordinary or not 
generally replicable. 

We are intrigued by the positive treatment effect on participants' 
satisfaction with teaching. This finding was not obtained in the prior eval- 
uation of ECM. One reason for this may be that discipline skills were 
substituted for problem-solving skills in the second version of ECM. Since 
maintaining order in the classroom was rated by teachers as very to extremely 
important, we speculate that providing them specific discipline techniques 
enhanced their feelings of control over classroom events. While it is 
important not to ascribe undue significance to this finding, certain aspects 
of ECM may hold promise for the development of prevention approaches that 
improve the environment for teachers. Insofar as teachers are the agents 
of change for most school-based prevention programs, this would seem a worth- 
while endeavor by virtue of its ramifications for students. 

In sum, ECM failed to produce the hypothesized pattern of findings. 
However, teachers who took the training were found to be more satisfied with 
teaching at the posttest than the control teachers. The present study will 
continue for an additional year. 
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